1 link tagged with all of: calibration + classification + benchmark + ai evaluation + llm
Click any tag below to further narrow down your results
Links
This page describes the Artificial Analysis Decision Index, a weighted benchmark that scores small "decision models" on fast classification tasks like routing support tickets and rating urgency. It combines six evaluations, from decision-making and tool calling to hallucination and answer invariance, into one composite score.
- Decision models answer every question in a single call with a probability for each option, which keeps them cheap and fast and makes calibration measurable.
- The index weights AA-Paperclip (document extraction as one decision) at 30%, with five other evaluations at 10โ15% each.
- Accuracy counts the top-probability option as the answer, and the page also reports Brier scores, time per task, and cost per 1M tasks.